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ABSTRACT 

The Self-Instructional Materials Project represents a 
significant step toward the development of a documented, replicable 
curriculum for medical education. It accomplishes this by carrying 
out the following generally accepted recommendations in the 
preparation of instructional materials: 1) it provides a useful 
technical manual — the Directory of Self-Instructional Materials — to 
accompany the materials it develops; 2) it regularly revises its 
materials; 3) it offers continuous dissemination of new materials; 4) 
instructional objectives are stated in detail; 5) objectives are 
operationally specified in terms of student behavioral responses; 6) 
the value of objectives is substantiated; 7) learning activities are 
directly related to objectives; 8) activities are sequenced to 
maximize student learning; 9) evaluation strategies are reported; 10) 
student target groups are identified; 11) validity data are cited; 
12) internal and external evidence are distinguished; 13) formative 
and summative evaluations are conducted; 14) both intended and 
unintended outcomes are reported; 15) utilization procedures are 
specified; and 16) basic and supplementary materials which are needed 
are identified. (PB) 
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TOWARD A DOCUMENTED OVWUCULUM 

i 

by 

Rita B. Johnson, Ed-D. 

In 1967 Louise L. Tyler and M. Frances Klein published Recommendations 
for Curricular and Instructional- Materials (D in response to the need for 
a curriculum document similar to APA's technical recommendations for psy- 
chological tests or to the mental measurements yearbook. It formulated 
some important recommendations which might be used as technical standards 
in evaluating medical curricula. 

Five years later, the Self-Instructional Materials Project, spon- 
sored by the Southern Medical School Consortium, addressed itself inten- 
tionally to the above recommendations. Stuart R. Johnson and Rita B. 
Johnson, et~al, published the Directory of Self-Instructional Materials 
in Medical Education (2) and attempted to document each item in its listing 
of one hundred and forty-eight self-instructional units. 

The purpose of this paper is to show how this Directory, the first 
of its kind, attempts to carry out a few of the recommendations suggested 
earlier by Tyler and Klein. 
Categories of Reco mmendations 

The recommendations formulated by Tyler and Klein were grouped into 
the following categories: 

I. General 

II. Specificat nms 

III. Rationale 

IV. Appropriateness 

V. Effectiveness 

VI. Condition:* ; . 

VII. Practical My 



Only those principles which were implemented in the Directory of 
Self-Ins tructional Materials will be described. Out of 25 recommendations, 
17 are discussed below. 

I. General . Into this category fall three recommendations that are 
general in nature: 

Gl. "When ctuuiicaZujn <md triA&iuctZonaZ matQAAJxXA ojvl developed, 
they AkouZd bz accompanied by a t&chyiLcaJL manuaJL tlicut makeA 
zvvtij naaAonahlfL z^^oht to faottou) the /iccormmdcutiayiA hi thu> 
docLLmzvit. " 

G2. "The. ciuVilciiLujn cuid hihtnixoXiowoJi matQJtiaLi tkouZd b& 
kslvXaqA at appropriate tntzAvaZb." 

G3. v ?/tov^tom AhouJtd be made continued dtb^emlviatLon. o£ c 
nojA) rnateJiLatb 9 nm appro acJizA , md new/ AtudLLeA*" 

Discuss iont In 1971 the deans of 28 southern medical schools formed 
a consortium to stimulate the development and exchange of self-instructional 
materials. Since then 1500 health sciences faculty have been trained by 
the Self-Instructional Materials Project to produce and revise self-instructional 
packages. . * 

This Project published its first edition of the Directory of Self- 
Instructional Materials in Medical Education . It lists materials cur- 
rently under production and gives information about the products. The 
Directory therefore functions as a technical manual. 

Decisions regarding what information was to be listed in this Directory 
were made on the basis of requests from samples of the user population. 
Over 250 workshop participants (faculty members who were producers of 
materials) were asked > "What would you find useful by way of information 



iii a document which would list other people 1 s products? What would you 
like to know about such products?" 

Responses were overwhelmingly in favor of the kinds of items listed 
in Tyler and Klein's Recommendations . This gave project staff reason to 
believe that such items would be helpful to other product users in medical 
education and would be appropriate for listing in the first project directory, 

All information in the Directory is now computerized, making easy access 
to a continuous updating and revision process. Pages can be quickly added 
or deleted. Information on any given page may be rapidly changed as 1) the 
producer acquires new evaluation data regarding the effects of his product 
on students; or, 2) the producer revises an existing package based on evalu- 
ation data he has already received. 

Specifications » In t his category are ^commendations referring to 
outcomes, since no definitive evaluation of curriculum can be ac- 
complished unless there are objectives: 

51. "T/ie. manual &kouZd AtcubL hx doJxuJL tliz objacXlvte 

52. "Objectives Akoiild be ^pzol^lnd opoJicubioYiallij , l.z. y l\xdhxd<L 

balxoxloh. ti&ypotete o& Atudwii*." 
Discussion : Table I (next page) is an entry 4:aken from the Directory 
of Self-Instructional Materials . It shows the cognitive objectives listed 
for a particular package entitled, "Diabetes Mellitus Complicating Pregnane;/' 
(P. 50). Note that the objectives are stated in terras of what the students 
are expected to do as an outcome of instruction. Note, also, that affective 
(i.e., attitudinal) measures are included as well, suggesting that affective 
objectives were important to the design of the package, (e. g. , items 1 to 4, 
Student Evaluation.) 
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TABLE I 
OBSTETRICS 

Diabetes ftlellitus Complicating Pregnancy (c!972) 2100-0002 



Luther Talbert, MX). 
North Carolina 

Students: Clinical \ 
Working time: 20 min. 
Availability: Contact 

Project Headquarters. 
Notes: Contains bibliography. 



FORMAT: 
Typescript 
Workbook 



Atttfude 
Feedback 
Practice 
Objectives 
Post-Test 



Version 3 
U^*. Version I 



OVERALL OBJECTIVES: 

Given a pregnant patient v/ith diabetes mellitirs, a description of her climca] course ar>d laboratory 
data, decide on a correct course of therapy; that is, prenatal management of her diab-tes and 
pregnancy as v/ell as timing of delivery. 

ENABLING OBJECTIVES: 

1. List four deleterious effects of maternal diabetes on the course of pregnancy. 

2. Describe the common problems encountered in the neonate of the diabetic mother 

3. Describe the effect of pregnancy on diabetes mellitus. 

4. List two hormones which contribute to increased difficulty in management of diabetes in A he 
pregnant patient. * c 

5. List three metabolic effects of human placental lactogen. 

6. Describe the effect of maternal insulin shock and keto-acidosis of the fetus. 
7: Define pre-diabetes and describe a method of diagnosis. 

8. Describe the I.V. GTT and its interpretation. 

9. You should be able to describe one method of classification of diabetes as related to 
pregnancy. 



STUDENT EVALUATION:* 

1. I thought this package wa& 

. too easy. 

1 too long. 

2 too short. 

1 rather difficult. ' 

5 fun. 

S educational. 

2. I would like to see: 

10 more material presented like this. 
1 this material presented a different way. 
.all of the clerkship presented this way. 



/ 



3. This package: 

5 contained questions that were too easy. 

v '^s too Mickey Mouse for medical students. 

7 really taught me a lot. 
bored me. 



4. After reading this package,. 

. 2 1 will probably want to learn more about the subject. 

2 I V/HI go to the library and read some of the references. 

3 I^eel that the package contained ali of the mat-rial I need to know about 

diabetes in pregnancy. 

(Continued on next page) 
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Number of Students Responding: 14 \ 
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III. Rationale . These items cover statements dealing with the process 

and reasoning involved in the choice of objectives, subject matter, 
etc. : 

Rl. "The value, o& tliz objzcJxvu nvjAt bz AubAtantiat&d." 
R5. "Lza/ovbig oppoh,iLi\vubLoA [FaIwa, toxtb, laboh.aXon.ij ma£2Jila£A r 
ztc. ) tkouZd be (WizcHy hzJLc&id to th.z behav<Lo/L and content 
ofa tJiz ApzosLfaiid obje-ctLveA 
R6. " LaaJUfiLng oppoh&utiUXQA mubt be, AO anAangtd &iat tko, bohavZon. 

o& tlw Atudkut aa dzvalopzd." 
R7. n Tkz \zZvid o& zvaZimtLon At/iat&gtj uAzd am dzveJiopivLQ tliz 

hx^tmiztioviaJL ma&tJiiaXA muAt be h.dpoht&d." ' 
Discussion : In Table II entitled, "How To Use This Directory" (next 
page) mention is made of a "graph found on the right hand side of each entry 
in the Directory . This graph tells the reader whether or not a given package 
has these specific self -instructional features: 



Post-Test: 



Is there a test for students at the end of the 



package which is consistent with the objectives? 



Objectives : 



Was the package designed for specific behavioral 



objectives? 



Practice: 



Are there frequent practice exercises included 



irf the material, and are they arranged so that the 



student practices new behavior in a graduated 



sequence? 



Feedback: 



Does the student get knowledge of results regarding 



his practice so he can correct his performance? 



UBLE II 



HOW TO USE THIS DIRECTORY 



Criteria for Listing of Packages: 

Selection of packages this Directory was based on these criteria: 

1. A copy of the package was in the Project library, and 

2. The package- met the minimum criteria for self-instruction; i.e., it contained at least 
a post-test, frequent practice and feedback. 

In addition, each package ideally was designed to meet specific objectives, including 
attitudinal. 

Arrangement of Materials Listed: 

There are two parts to this Directory. The first part, on white paper, lists packages for 
the medical school curriculum. The basic arrangement is by subject, alphabetically. 
Sub-arrangement is alphabetical by title. The second part of the Directory, on yellow 
paper, lists packages for the allied health areas. Basic arrangement is alphabetical by 
specialty area, such as! Dentistry, Nursing, Physical Therapy, etc. Hie sub-arrangement is 
alphabetical by title. An index to ail subjects, authors and titles is provided for your 
convenience. 

Information about each package is presented graphically in the format shown here. The 
objectives for the unit and any available try-out data will be given on the same* page 
below the entry. ■ 



Self-instructional Features: 



Package does have 
these features. 

Y . . 




Package does not have 
these features. 



ANATOMY 




The Bones of the Foot (1971) 0201 

John Doe 
^ Southern Medical School 
^ Students: 

Working time: 
Availability: Contact 



-0005 
FORMAT: 



Typescript 
11 slides 
Audio cassette 



Producer or Coordinator 



directly. 



OBJECTIVES: 



STUDENT COMMENTS: 



TRY-OUT DATA: 



Revision history: 
This package is in 
second edition.. 



School where package was developed. 



i 



VI 



Attitude: Is there a measure of attitudinal change 
taken at coitrpletion of the package? 
Version 1, 2 or 3: How many times h v as this package been revised? 
(To illustrate in Table II John Doe's unit has no attitudinal measure 
although it does include all of the other features. The graph indicates 
this material to be second version.) 

In addition to the above features, the format of the learning materials 
is described. (For e.g., John Doe T s unit consists of a typed script, 11 
slides and an audio-cassette). The reader can* determine for himself if the 
learning opportunities are directly related to the behavior and content of 
the objectives which are given. 

Finally, the try-out data to be reported in the next Directory will 
include peer-review comments from faculty members who are urged to substantiate 
the value of the objectives for their specific content area. Peer-review 
sheets forwarded by Project Headquarters and completed by these specialists 
include questions such as, "Was the material appropriate for the target 
group? 11 " "Is the content useful?" "Was the information accurate?" "Is the 
data up-to-date?" "Are the objectives and post- test items important?", etc. 
IV. App rop ria t enes s « This category includes statements having to do with 
the kind of learner for whom the material is developed, so that evalu- 
ation can be done in terms of learner characteristics. 
Al. "Thz kind o& 6tudzn£ faon whom tiid mcitQJhiaLb a/ct deA-igntd ^iwuld 
be >6pacu^£c{. 

Discussion : Referring again to Tables I and II we note that the target 
group of students is broadly defined. (e.g., in Table I, Dr. Talbert T s 
unit is designed for clinical students.) Categories of students in the 



current Directory include pre -health, p re-clinical, and clinical. Target 
groups in the related health areas include "Patients with Diabetes, 11 
"Family Nurse Practitioners, " "Radiology Scheduling Clerks," "Dental Patients, 
"Medical Records Science Students," etc. 

It should be mentioned that as additional evaluation data is collected, 
target groups may shift. For example, a set of materials originally designed 
for clerks in one third year obstetrics-gynecology course were used in a 
hospital where residents and interns were working nearby. The packages were 
deemed useful as "refresher material" so that now these same packages are 
made available to the fifth and sixth year students who need rapid review 
of familiar material. 

V» Effectiveness . In this group are statements about characteristics 
. -..-.which determine how the._curriculum was evaluated. 

El. "Manual* should cite 6olumlqa a* available. evidence to document 

any clalM made, about e£fieotlvene/>& and e^Juclevicy ." 
E2. "M<mual& should cleanly (tUtingul^k between kinds oft evidence 

pticAevited about eUectlvenu*-. [a] IvvtennaJL evidence, (fa) extetoial 
evidence.. lnteJur.aJL Jue&eAA to fceatusiu tievealed tivtough vUuat 
impaction oq the. mate/tlaZb. External tiefate to tfty-outs, tie- 
vlblonk , etc." 

E3. "Evaluation should be. utilized token appwpJtlate In tlie psloc&u 
o& InAt/utctional de.veZopme.vit. KIao evaluation should be used 
token mateJilaLs cjie completely developed. 11 

E4. tf E^ecJtiveneA6 o& pHjognxxm should be Keponted In tern* o( x p/togjtam 
objective* as well cos LuiMit&ided outcomes .* 



Discussion : This first Directory lists attitudinal responses (both 
intended and unintended) from students and faculty. The exact sources of 
such data are available to anyone who contacts Project Headquarters in 
Chapel Hill. In many cases the materials are sent to several institutions 
beforehand and the numbers of students trying out the package in each 
sample are shown. 

In future editions of the Directory , cognitive achievement will also 
be reported for each entry. A fraction will apppar by the graph showing 
the number of students who meet the cognitive objectives, divided by the 
total number in the sample (e.g., 9/10). 

As Table II indicates, all evidence from students consists of try-out 
data, i.e., come from users who have actually tried out the material and 
attempted to pass the post-test. Currently, the data reported come from 
comments or reactions to questionnaires. 

At this early stage of development, the data are used in the formative 
revision process. Project staff urge producers to revise continuously 
until objectives are met. It is assumed that as the project continues 
more and more materials will be completely developed so thar summative 
evaluation data can be reported. For example, evaluation studies are 
currently being conducted at one university on a few well-documented 
materials in obstetrics-gynecology and pharmacology to determine the 
effect of* these packages on performance during national board examinations, 
(reported elsewhere in Journal of Medical Education) 

VI. Conditions . In this category are items having to do with the known 

conditions of use so that the user can determine whether his situation 
is similar to the setting described. 
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C4. u ?/ioc<Ldute& and aJUu^QOmmU o£ vutlZJxhig tkd matwiaX/, fan. 

dz&inzd bamplzA AiudzwtA wuAi be ^pect^ed." 
Discussion : Each entry specifies the format of the materials and 
the approximate amount of time it takes the average user to complete the 
material. Special conditions of use are frequently spelled out, such as 
in the entries below: 

"TlvU paokoX /izquAJceA a paJvtYiQJi." [P. 40) 
"TfuA package, trn^ dzAlgnoA ion. one pahtlcaloA gn.oup 
oh AtudzYitA. Hom\J2A, It may be u*ed oa gl modoZ. 
tt/iLt InoltideA AtjnopAQA o£ po^lblz pn.ojzct^. u 
VII. Practicality . These recommendations relate to factors basic 

to use in a particular setting, e.g., cost of materials, building 
facilities, etc. 

PI. r 'T/te QiLidz muAt hidlzatz vMck liu&dic£LoyictZ matoJoia^A 
aXz SLdqu-Oizd. Wke/te uipplmznta/tij mcLtzJdLaJlA ojiz to be 
uAzd, tliQAZ Ahoiitd be dtecAsthzd." 
Discussion : Supplementary equipment, materials and extra requirements 
are added to each entry in the Directory whenever necessary to make the 
package complete for use by others. For example, current entries include 
the following: 

"Tkz M&cUcaZ LzttoJc R^ojcznczA Handbook, 3a:mamj .1971 , 
U u,5ec£ oa toxt." {P. 67) 

"Vou will need aa eqcupment: (?) Polaroid Lmd CaneAa, 
lAodzl 540 o/i 350 (2) OlymptjA &J>beJu>p£Lc QAophago- 
QCl6tAQ6COp<L. 1! (P. 18) 
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"ModeJL Pztvlb a6e.cf vttih package,." (P. 47) 

"\KateJiLaLk nzzdzd: &aZad dsu>A&<oiQ} apple. Mitii a bfujuohz) 

AutuAZ nzzdJLz and nzzdtd hoZdeA; bdaJLpzL; 2x2 Aponge.) 

titeuz tfo/Lcep4„" (P. 727) 

Summary 

No attempt has been made to demonstrate the overall effectiveness 
or quality of materials which currently appear in the first edition of the 
Directory of Self-Instructional Materials . It has been shown instead that 
this document does carry out some of the technical recommendations for 
curriculum materials suggested by Louise L. Tyler and M. Frances Klein 
in 1967. 

This first Directory published by the Self-Instructional Materials 
Project is still in process of refinement. However, it is believed that 
it represents a significant beginning step towards developing a documented, 
replicable curriculum in medical education as far as products are concerned. 
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